In the Claims 



The status of claims in the case is as follows: 



1 1. [Currently amended] A cache coherency system for a 

2 shared memory parallel processing system including a 

3 plurality of processing nodes, comprising: 

4 a single multi-stage communication network for 

5 interconnecting said processing nodes , said network 

6 including a dual priority switch at each node for 

7 selectively operating in normal low priority mode and 

8 camp-on high priority mode ; 
9 

10 each said processing node including a unique section of 

11 shared memory which is not a cache; 

12 each said processing node including one or more caches 

13 for storing a plurality of cache lines; 

14 a cache coherency directory which is distributed to 

15 each of said nodes for tracking which of one or more of 

16 said nodes have copies of each cache line; and 
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an adapter for storing changed data immediately to said 
unique section of shared memory regardless of which of 
said nodes is changing the data and which of said nodes 
includes the section of shared memory to be changed, 
such that said shared memory always contains the most 
recent data according to a two hop process including in 
hop 1) a requesting node requests most recent data of a 
home node, and in hop 2) said home node immediately 
returns said most recent data from its shared memory to 
said requesting node . 
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2 . [Withdrawn] A shajred 
system including a plural 
comprising : 



a multi-stage communi 



said processing no^i 
plurality of self 



rou 



id 



middle and last stagf 
plurality of switch 
outputs, each of sa 
switch coupled to a 
said switches, switc 
switches including 



memory parallel processing 
ty of processing nodes, 



cation network for interconnecting 
f g said network including a 
itlng switches cascaded into first, 
s, each said switch including a 
:.nputs and a plurality of switch 

switch outputs of each said 
different switch input of others of 
outputs of said last stage 
twork output ports, and switch 



ne 
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inputs of said firpt stage switches comprising network 
input ports; 



each processing no 

a network adc. 
messages wit 



de including: 

pter for transmitting and receiving 
respect to other processing nodes 



over said network; 



19 



a local processor; 
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at least ore private write- through cache; 

a sect ioiVcfi shared memory organized into a 
plural ity-"of Vache lines, each cache line 
including ope or more addressable memory 
locations; 

a cache coherency directory for tracking which of 
said nodes have copies of each cache line; 

said local processor at a first processing node being 
operable for writing data to said private cache at said 
first node, as the! same data is written to either 
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shared memory at said first node or sent over said 
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network for writing 



to the shared memory and private 



cache of a second processing node. 



3 . [Withdrawn] The 
system of claim 2, where 
divided into first and 
for storing unchangeable 
storing changeable data 

4 . [Withdrawn] The 



shared memory parallel processing 
:.n said section of shared memory is 
second portions, said first portion 
data, and said second portion for 



shared memory parallel processing 
system of claim 3, said cache coherency directory for this 
processing node listinc 
nodes have accessed fco 

second portion of sfra^eW-irtemory at this processing node. 



m which nodes of the plurality of 
Dp.es of said cache lines in said 
eo-tffemc 



5. [Withdrawn] Ttie shared memory parallel processing 
system of claim 4, wherein each said processing node is 
operable for reading, storing, and invalidating the shared 
memory at any of said plurality of processing nodes 
selectively by transmitting and receiving messages over said 
network, a first message type for requesting the read of a 
cache line, a second message type for returning the 
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requested cache line, a (third message type for storing a 
cache line, and a fourtlji message type for invalidating a 
cache line. 



6 . [Withdrawn] The 

system of claim 5, sai 



shared memory parallel processing 
network adapter further comprising: 



3 
4 
5 



a first buffer for transmitting to said network shared 
memory read command messages of said first message type 
and said second message type; 



6 
7 
8 



a second buffer jfqr transmitting to said network shared 
memory store c/SnJma^id messages of said third message 
type; 



9 
10 
11 



a third buffer Ifor transmitting to said network 
invalidate messages for said cache coherency directory 
of said fourth (message type; 



12 
13 
14 



a fourth buffet for receiving from said network shared 
memory read command messages of said first message type 
and said second message type; 



15 



a fifth buffer for receiving from said network shared 
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invalidate messages 



memory store command messages of said third message 
type ; and 

a sixth buffer for deceiving from said network 



of said fourth message type. 



7 . [Withdrawn] A sh 
system, comprising : 

a plurality of 
memory, at least o 



for said cache coherency directory 



c red memory parallel processing 



nodes, each node including a node 
e cache, and a memory controller; 
a multi-stag^f^witching network for 
interconnecting (sa^a^pjpeessing nodes, said switching 
network including |a plurality of self-routing switches 
cascaded into fir^t, middle and last stages, each said 

a plurality of switch inputs and a 
^h outputs, each of said switch 
outputs of each said switch coupled to a different 
switch input of others of said switches, switch outputs 

e switches including network output 
inputs of said first stage switches 
comprising network input ports; 

a system menory distributed to said node memories 
of nodes and accessible by any node; 



switch including 
plurality of swit 



of said last stag 
ports, and switch 



of said plurality 
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each said node memdry being organized into a plurality 
of addressable word locations; 

said memory controller at this node operable for 
performing local memory access to the portion of system 
memory at this node and for performing remote memory 
access over said network to the portion of system 
memory at other nodes; and 

a cache coherency controller at this node being 
local memory accesses and remote 
data stored in a word location of 
said node memory dt this node for caching accessed data 
in the cache of tlvfs node and for communicating data 
for assuring ca^^l^coi^B^ncY throughout said system 
over said network 



The 



responsive to both 
memory accesses to 



8. [Withdrawn] 
claim 7, said system rr 
portions to each said 
being further sub- 
storing data that is 
for storing data that 



divided 



shared memory processing system of 
emory being distributed in equal 
node memory; and said node memory 
into a first memory section for 
changeable and a second memory section 
s unchangeable. 



9. [Withdrawn] Th<> shared memory processing system of 
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2 claim 7, further comprising node indicia for uniquely 

3 identifying each node. 



1 10. [Withdrawn] The shared memory processing system of 

2 claim 7, said cache coherency controller further comprising: 



3 
4 
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an invalidation directory for storing a list of node 
indicia identifying those nodes having accessed a copy 
of each said cache line of node memory since the last 
time the cache line was changed. 



1 11. [Withdrawn] tYiM "feh^fed memory processing system of 

2 claim 10, said cache coherency controller further 

3 comprising: 



4 
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an overflow directory for expanding said invalidation 
directory when the list of node indicia for a cache 
line becomes too [Long to contain entirely with said 
invalidation directory. 



1 12. [Withdrawn] A [shared memory parallel processing 

2 system, comprising: 
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a plurality of nodes 
memory, at least one 



, each node including a node 
cache, and a memory controller; 



ing network for interconnecting 
s, said switching network including 
routing switches cascaded into 
ist stages, each said switch 
ty of switch inputs and a plurality 
each of said switch outputs of each 
to a different switch input of 
others of said switches, switch outputs of said last 
stage switches inc./Qding network output ports, and 

ir3-t- stage switches comprising 
; and 



a multi-stage switch 
said processing node 
a plurality of sel 
first, middle and 1 
including a plurali 
of switch outputs, 
said switch couplec. 



switch inputs of 
network input portb 




a network adapter (responsive to a node connection 
request for establishing a connection path to a target 
node, first by attempting to establish a quick 
connection path across a plurality of segments of said 
switching network no said target node, and upon 
determining any one! of said plurality of segments is 



not available, issu 
said target node. 



ng a camp-on connection request to 
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1 13. [Withdrawn] The soared memory parallel processing 

2 system of claim 12, furtherr comprising: 

3 said plurality of nodes each coupled to one of the 

4 network output ports and to one of the network input 

5 ports; 

6 each node further including: 

7 receive means nor receiving a data message; and 

8 send means for sending a data message across an 

9 n- stage switching network from a local node to a 

10 remote node, /skid send means generating said 

11 connection request including n sequential 

12 connection commands, each sequential connection 

13 command selecting one of said plurality of 

14 connection segments for each of the n switch 

15 stages of said network. 

1 14. [Withdrawn] The shared memory parallel processing 

2 system of claim 12, eacm said switch being responsive to 

3 node connection requests and camp-on connection requests for 

4 establishing connection segments from any switch input port 
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15. [Withdrawn] The 
system of claim 14, each sa 



shared memory parallel processing 
Ld switch further comprising: 



a data bus for transf erring said data message; 



a rejection control li 
sending node a rejecti 

an acceptance control 
sending node the accej 
request ; 



ne 



on 



Line for signalling back to said 
:ance of a camp-on connection 




a valid control line 
node the activation of 



fpr receiving from said sending 
a node connection request; and 



for signalling back to a 
of any connection request; 



5 to any switch output ports. 

1 
2 
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11 a camp-on control line 

12 node the activation of 



for receiving from said sending 
a camp-on connection request, 



1 16. [Withdrawn] A bi-directional network adapter for 

2 interfacing a local node of a shared memory parallel 

3 processing system to a multi-stage switching network for 

4 communications with a remot s node, each said node including 
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5 a node memory including a changeable portion and an 

6 unchangeable portion, and a [node cache; said network adapter 

7 comprising: 



8 
9 
10 



a plurality of send buffers for storing and forwarding 
data messages from saifd local node to said remote node 
ove r said ne t wor k , anc 



11 
12 
13 
14 



a plurality of receive buffers for storing and 

forwarding a plurality of data messages from said 

remote node to said Ipcal node over said multi-stage 
network; 



15 



said data messages [includii 



16 
17 
18 



an invalidation message for invalidating a cache 
line that was accessed by a remote node after said 
cache line has changed; 



19 
20 

21 
22 



a read request 
cache line from 

a response mess< 
the network to « 



essage for requesting access of a 
a remote node; 

ge for returning a cache line over 
remote node that has previously 
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requested data by 

a store message st 
remote node . 



a read request message; and 



oring a changed cache line to a 



17. [Withdrawn] The network adapter of claim 16, said 
data messages further including a message header comprising: 



message type different 



destination node indi 
receiving said data 



:_ation indicia; 



cia for identifying a node for 
me|ssage over said network; 



source node indicia /for^ identifying a node for 
transmitting said data] message over said network; 



8 
9 



message length indicia for defining the variable number 
of words included in skid data message; 



10 
11 
12 



memory area indicia for defining whether memory words 
included in said data rfiessage are read from said 
changeable area; 



13 



time indicia for defining the time of generation of 
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said data message; and 



15 
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memory address indicia (for defining the address 
location in memory of t(he memory word included in said 
data message. 



1 18. [Withdrawn] The network adapter of claim 17, said 

2 send buffers further comprising: 
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a read send FIFO for storing and forwarding read 
request messages and response messages from said local 
node to said remote 



an invalidation send 
invalidation messages 
remote node; 




a store send FIFO fog: IgcoftTiq and forwarding store 
messages from said lodal node to said remote node; and 



and said receive buffers farther comprising 



a read receive FIFO 
request messages and 



IFO for storing and forwarding 
from said local node to said 



f<pr storing and forwarding read 
response messages from said remote 
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node to said local 

a store receive FI 
messages from said 



node ; 



FD 



for storing and forwarding store 
remote node to said local node; and 



an invalidation redeive FIFO for storing and forwarding 
invalidation messages from said remote node to said 
local node. 



1 19. [Withdrawn] The) network adapter of claim 18, further 

2 comprising: 
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a send FIFO selection means for prioritizing the 
selection of a dati^me^&a^e from one of said three send 
FIFO means for transmission to said network by first 
selecting data messages from said invalidation send 
FIFO and thereafter alternatively selecting data 
messages from saiq read and store send FIFOs; 



9 
10 
11 
12 



a receive FIFO selection means responsive to said 
message type differentiation indicia for selecting one 
of said three receijve FIFO means for storing a data 
message received frpm said network; and 
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said network adapte 
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connection request 
to a target node, 



for establishing a connection path 
irst by attempting to establish a 
quick connection palth across a plurality of segments of 
said switching network to said target node, and upon 
determining any on* of said plurality of segments is 
not available, issuing a camp-on connection request to 
said target node 



20. [Withdrawn] A n 
a shared memory para 
including a node pr 
an I/O adapter, said S} 
switching network for 
and a plurality of remcj>t 
a changeable portion 
memory controller compti 



and 



jc being responsive to a node 




^mory controller for a local node of 
processing system, said node 

node memory, a node cache and 
tern including a multi-stage 
Communications amongst said local node 
e nodes, said node memory including 

an unchangeable portion; said 
sing : 



first means respoifs 
for access to a 
node cache of saic 



ive to a request by said processor 
memory word for first accessing said 
local node; and 



12 



second means respoisive to said first means being 
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13 unable to access saip memory word in said node cache 

14 for accessing said memory word selectively from a cache 

15 line in said node memory or remote memory and storing 

16 said cache line to said node cache. 



1 21. [Withdrawn] The memory controller of claim 20, 

2 further comprising: I 

3 remote fetch interrupt means for issuing an interrupt 

4 signal to said node processor upon determining that a 

5 requested memory w6rcl is located in remote memory for 

6 causing said node processor to switch from a first 

7 instruction streaflf ttaeeeduo a second instruction 

8 stream thread. 



1 22. [Withdrawn] Thf memory controller of claim 20, 

2 further comprising: 



3 
4 
5 
6 
7 



data message generation means responsive to a request 
from a remote node! for accessing a cache line 
identified by a remote request read address for 
generating a read Response message to return the 



accessed cache line 



to said remote node, said read 
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message different 
read request mes 



Lation indicia for defining said 
s|age type ; 



destination node 
segment of said r 
memory word; 

source node indict 
the local node; 



indicia equal to the sector 
ode memory for said addressed 



message leng 
request message $s 
header only; and 

memory address i 
address of said 



said data message 
delivering said read 
FIFO of said network 
network and the remote 
destination node indie 



ing a message header comprising 



ia set to the node ID number of 




for defining said read 
being comprised of said message 



indicia for specifying the memory 
riiemory word; 



gene rat 



ion means further operable for 
response message to a read send 
adapter for transmission to said 
node selected by said 
ia. 
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23. [Withdrawn] The mefnory controller of claim 20, 
further comprising: 



an invalidation direc: 



tory; 



cast -out means for deleting a cache line from said node 
cache when said cache is full to provide space for a 

stored to said cache; and for 
of the deleted cache line to said 
invalidation directory to indicate said node no longer 
has a copy of said cache line. 



new cache line to be 
sending the address 



rhe nefno: 



24. [Withdrawn] The nefnory controller of claim 23, 



further comprising: 



cast -out means delet: 



cast -out message generation means responsive to said 



ing a cache line addressed to a 



remote node for generating a cast -out message to said 



remote node to send 



the cast -out address and the local 



node ID number to said remote node over said network; 



cast -out message rec|e 
cast -out address and 



iving means for delivering a 

the source node ID number from the 
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message header of a c^st-out message to said 
invalidation directory 



1 25. [Withdrawn] The mefmory controller of claim 20, 

2 further comprising: 



3 
4 
5 



cache copy update means for sending cache update 
messages to update corresponding cache lines all remote 
nodes having copies jof a changed cache line; and 



6 
7 
8 



cache update message receiving means for replacing a 
cache line of data with an updated cache line of data 
received from a ifempne node. 
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26. [Withdrawn] 



The bi-directional network adapter of 



claim 16, said data mess 

a cast-out message 
directory entry at 

a cache copy update 
changed cache line 
having copies of sa 



ages further comprising: 

for invalidating an invalidation 
a remote node for this local node; 

message for updating copies of a 
at this local node at remote nodes 
id changed cache line; and 
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a node indicia assignment message for sending a 



different node number 
of the system. 



bo each of the plurality of nodes 



27. [Withdrawn] A method 
for a local node of a shared 
system, said node including 
a node cache and an I/O adajbt 
multi-stage switching netwo: 
said local node and a plura 
memory including a changeab 
portion; the method compels 



for operating memory controller 
memory parallel processing 
a node processor, a node memory, 

er, said system including a 
k for communications amongst 
ity of remote nodes, said node 
e portion and an unchangeable 
the steps of : 




responsive to a request 
a memory word, accessing 
node; and thereafter 



rs t 



responsive to said fi 
said memory word in sai 
memory word selectively 
memory or remote memory 
said node cache. 



by said processor for access to 
said node cache of said local 



means being unable to access 
d node cache, accessing said 
from a cache line in said node 
and storing said cache line to 



END919970080US2 



S/N 09/394,564 



1 28. [Withdrawn] The fnethod of claim 27, further 

2 comprising the step of: 
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issuing an interrupt signal to said node processor upon 
determining that a| requested memory word is located in 
remote memory for (causing said node processor to switch 
from a first instruction stream thread to a second 
instruction stream thread. 



29. [Withdrawn] A 
network adapter for i 
memory parallel proc 
network for communi, 
node including a n 
and an unchangeable 
the steps of : 



at 



i>de 



plural 



operating a 
forwarding data 
remote node over 



•nethod for operating bi-directional 
interfacing a local node of a shared 
sTng^ system to a multi-stage switching 
iojjs with a remote node, each said 
emory including a changeable portion 
pdrtion, and a node cache; comprising 



ity of send buffers for storing and 
messages from said local node to said 
said network, and 



operating a plurality of receive buffers for storing 



and forwarding a 
remote node to s 



t 



plurality of data messages from said 
id local node over said multi-stage 
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network; 



said data messages including 



an invalidat 
line that wals 
cache line Y 



request 



a read 
cache line i: 




ion message for invalidating a cache 

accessed by a remote node after said 
as changed; 



message for requesting access of a 
rom a remote node; 

ssage for returning a cache line over 
^) a remote node that has previously 
by a read request message; and 



a store message storing a changed cache line to a 
remote node . 

30. [Withdrawn] Th^ method of claim 29, further 
comprising the steps ofl: 



3 
4 
5 



operating a read send FIFO for storing and forwarding 
read request messages and response messages from said 
local node to said remote node; 
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operating a store 3e 
store messages from 
node ; and 



operating an inval 
forwarding inval id^t 
to said remote nodes 



nd FIFO for storing and forwarding 
said local node to said remote 



dation send FIFO for storing and 
ion messages from said local node 



operating a read receive FIFO for storing and 
forwarding read request messages and response messages 
from said remote node to said local node; 



operating a sto 
forwarding store me 
local node; and 




<5ceive FIFO for storing and 
sages from said remote node to said 



operating an inval i 
forwarding invalidate 
to said local node. 



Ration receive FIFO for storing and 
ion messages from said remote node 



31. [Currently amended] A method for operating a shared 
memory parallel processing system as a cache coherency 
system including a plurality of processing nodes, each said 
processing node including a unique section of shared memory 
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5 which is not a cache, comprising the steps of: 

6 interconnecting said processing nodes through a single 

7 multi-stage communication network , said network 

8 including a dual priority switch at each node for 

9 selectively operating in normal low priority mode and 

10 camp-on high priority mode ; 

11 storing at each said processing node a plurality of 

12 cache lines in one or more caches; 

13 distributing to each of said processing nodes a cache 

14 coherency directory; 

15 tracking in said cache coherency directory which of 

16 said one or more of said processing nodes have copies 

17 of each cache line; and 

18 changing said shared memory according to a two hop 

19 process including in hop 1) a requesting node 

2 0 requests most recent data of a home node, and in hop 

21 2) said home node immediately returns said most 

22 recent data from its shared memory to said requesting 

23 node , wherein changed data is stored immediately to 
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24 said unique section of shared memory regardless of 

2 5 which of said nodes is changing the data and which of 

2 6 said nodes includes the section of shared memory to 

2 7 be changed, wherein said shared memory always 

2 8 contains the most recent data. 

1 32. [Currently amended] A program storage device readable 

2 by a machine, tangibly embodying a program of instructions 

3 executable by a machine to perform method steps for 

4 operating a shared memory parallel processing system 

5 including a plurality of processing nodes, each said 

6 processing node including a unique section of shared memory 

7 which is not a cache, said method steps comprising: 

8 interconnecting said processing nodes through a single 

9 multi-stage communication network , said network 

10 including a dual priority switch at each node for 

11 selectively operating in normal low priority mode and 

12 camp-on high priority mode ; 

13 storing at each said processing node a plurality of 

14 cache lines in one or more caches; 



15 



tracking in a cache coherency directory which is 
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16 distributed to each of said processing nodes which of 

17 one or more of said processing nodes have copies of 

18 each cache line; and 

19 changing said unique section of shared memory according 
2 0 to a two hop process including in hop 1) a requesting 

21 node requests most recent data of a home node, and in 

22 hop 2) said home node immediately returns said most 

2 3 recent data from its shared memory to said requesting 

24 node , wherein changed data is stored immediately to 

2 5 shared memory regardless of which of said nodes is 

2 6 changing the data and which of said nodes includes the 

2 7 section of shared memory to be changed, wherein said 

2 8 shared memory always contains the most recent data. 

1 33. [Currently amended] An article of manufacture 

2 comprising: 

3 a computer useable medium having computer readable 

4 program code means embodied therein for operating a 

5 shared memory parallel processing system including a 

6 plurality of processing nodes, each said processing 

7 node including a unique section of shared memory which 

8 is not a cache, the computer readable program means in 
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9 said article of manufacture comprising: 

10 computer readable program code means for causing a 

11 computer to effect interconnecting said processing 

12 nodes through a multi-stage communication network , said 

13 network including a dual priority switch at each node 

14 for selectively operating in normal low priority mode 

15 and camp-on high priority mode ; 

16 computer readable program code means for causing a 

17 computer to effect storing at each said processing node 

18 a plurality of cache lines in one or more caches; 

19 computer readable program code means for causing a 

20 computer to effect tracking in a cache coherency 

21 directory which is distributed to each of said 

22 processing nodes which of said processing nodes have 

23 copies of each cache line; and 

24 computer readable program code means for storing 

25 changed data immediately to said unique section of 
2 6 shared memory regardless of which of said nodes is 

27 changing the data and which of said nodes includes the 

2 8 section of shared memory to be changed according to a 
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2 9 two hop process including in hop 1) a requesting node 

30 requests most recent data of a home node, and in hop 2) 

31 said home node immediately returns said most recent 

32 data from its shared memory to said requesting node, 

33 such that said shared memory always contains the most 

34 recent data. 

1 34. [Currently amended] A computer program product or 

2 computer program element for operating a shared memory 

3 parallel processing system including a plurality of 

4 processing nodes, each said node including a unique section 

5 of shared memory which is not a cache, according to the 

6 steps of: 

7 interconnecting said processing nodes through a single 

8 multi-stage communication network , said network 

9 including a dual priority switch at each node for 

10 selectively operating in normal low priority mode and 

11 camp-on high priority mode ; 

12 storing at each said processing node a plurality of 

13 cache lines in one or more caches; 

14 distributing to each of said processing nodes a cache 



EMD919970080US2 30 S/N 09/394,564 




15 coherency directory; 

16 tracking in said cache coherency directory which of 

17 said processing nodes have copies of each cache line; 

18 and 

19 storing changed data immediately to said unique section 

20 of shared memory regardless of which of said nodes is 

21 changing the data and which of said nodes includes the 

22 section of shared memory to be changed according to a 

23 two hop process including in hop 1) a requesting node 

24 requests most recent data of a home node, and in hop 2) 

25 said home node immediately returns said most recent 

26 data from its shared memory to said requesting node 

27 such that said shared memory always contains the most 
2 8 recent data. 

1 35. [Original] The cache coherency system of claim 1, 

2 further comprising: 

3 a shared memory including a first memory portion for 

4 storing unchangeable data and a second memory portion 

5 for storing changeable data; and 
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6 said cache coherency directory listing which nodes of 

7 said plurality of processing nodes have accessed copies 

8 of said cache lines in said second memory portion. 

1 36. [Original] The cache coherency system of claim 35, 

2 each of said plurality of processing nodes being operable 

3 for reading, storing, and invalidating said shared memory at 

4 any other of said processing nodes. 

1 37. [Previously presented] The cache coherency system of 

2 claim 36, further comprising at a first node of said 

3 plurality of processing nodes a memory controller 

4 selectively operable first responsive to a request for 

5 access to a memory word by first accessing the cache at 

6 said first node and, if said requested memory word is not 

7 available in said cache, selectively operable second for 

8 accessing said memory word selectively from said shared 

9 memory regardless of which of said nodes includes the 

10 section of shared memory being accessed, and storing said 

11 cache line including said memory word to said cache at said 

12 first node. 

1 38. [Previously presented] The cache coherency system of 

2 claim 37, said memory controller further being selectively 
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3 operable for deleting a cache line from said cache at said 

4 first node when said cache is full to provide space for a 

5 new cache line to be stored to said cache, and for sending 

6 the address of the deleted cache line to an invalidation 

7 directory to indicate said node no longer has a copy of said 

8 cache line. 

39. [Previously presented] The cache coherency system of 
claim 37, said memory controller further being selectively 
operable for sending cache update messages to update 
corresponding cache lines at all remote nodes having copies 
of a changed cache line and for receiving cache lines of 
data from remote nodes for updating the cache at said first 
node . 
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